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ABSTRACT 

Video  enhancement  has  played  very  important  roles  in  many  applications.  However,  most  existing  enhancement 
methods  only  focus  on  the  spatial  quality  within  a  frame  while  the  temporal  qualities  of  the  enhanced  video  are  often 
unguaranteed.  In  this  paper,  a  new  algorithm  is  proposed  for  video  enhancement. 

The  proposed  algorithm  introduces  new  temporal  constraints  and  combines  them  with  spatial  constraints  such  that 
both  the  spatial  and  temporal  qualities  of  the  video  can  be  improved.  Two  strategies  are  proposed  for  including  the 
temporal  constraints.  Experimental  results  demonstrate  the  effectiveness  of  the  proposed  algorithm. 
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INTRODUCTION 

Video  enhancement  is  of  increasing  importance  in  many  applications  such  as  medical  services, 
display  processing,  texture  analysis,  surveillance,  and  scientific  visualization  [2,  9-12]. 

Many  researchers  have  been  done  on  video  or  image  enhancement  [1-10].  Many  works  perform  enhancement 
based  on  histogram  equalization  (HE)  such  that  the  contract  of  the  image  can  be  improved  [1,  3-6].  Since  pure  HE  may 
often  lead  to  unnatural  affects  in  the  image,  various  HE-modification  methods  are  proposed  which  introduce  spatial 
constraints  to  reduce  these  unnatural  effects  [3-6].  Other  works  [3,  10]  select  some  appealing  images  as  the  examples  and 
perform  enhancement  based  on  these  example  images. 

However,  most  of  the  existing  enhancement  algorithms  only  focus  on  improving  the  spatial  qualities  with  a  single 
frame  or  an  image.  They  are  not  suitable  for  enhancing  videos  since  the  temporal  quality  consistencies  among  frames  are 
not  guaranteed. 

There  are  some  works  proposed  for  enhancing  videos  under  some  specific  applications.  For  example,  Liu  et  al. 
[10]  propose  a  learning -based  method  for  videoconferencing  where  frames  share  the  same  tone  mapping  function  if  their 
backgrounds  do  not  change  by  much.  Although  this  method  can  achieve  good  temporal  quality  in  video  conferencing 
scenarios,  it  cannot  be  extended  to  other  scenarios  since  there  are  many  videos  whose  backgrounds  or  contents  are 
changing  frequently.  Furthermore,  Mangiest  al.  [9]  uses  two  cameras  with  different  exposures  to  create 
high-dynamic-range  frames.  However,  this  method  has 

Specific  system  requirements  and  thus  cannot  be  easily  used  in  other  scenarios.  Therefore,  it  is  desirable  to 
develop  a  more  generalized  enhancement  algorithm  which  can  handle  the  temporal  quality  of  various  videos. 
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In  this  paper,  a  new  Temporal-Constraint-Based  (TCB)  Algorithm  is  proposed  for  video  enhancement. 
The  proposed  TCB  algorithm  introduces  new  temporal  constraints  and  combines  them  with  spatial  constraints  such  that 
both  the  spatial  and  the  temporal  qualities  of  videos  are  improved.  Two  different  strategies  are  proposed  for  defining  and 
integrating  the  temporal  constraints.  The  proposed  algorithm  provides  a  generalized  way  for  handling  the  video  temporal 
qualities.  Experimental  results  demonstrate  the  effectiveness  of  our  proposed  algorithm. 

There  are  two  main  methods  to  process  an  image  as  defined  by  the  domain  in  which  the  image  is  processed, 
namely  spatial-based  domain  and  frequency-based  domain.  Spatial  based  domain  refers  to  the  image  plane  itself, 
and  approaches  in  this  category  are  based  on  direct  manipulation  of  pixels  in  an  image.  Frequency-based  domain 
processing  techniques  are  based  on  modifying  the  spatial  frequency  spectrum  of  the  image  as  obtained  by  transform. 

Enhanced  techniques  based  on  various  combinations  of  methods  from  these  two  categories  are  not  unusual  and 
the  same  enhancement  technique  can  also  be  implemented  in  both  domains,  yielding  identical  results.  With  the  same  image 
processing,  a  lot  of  video.  Enhancement  methods  have  been  proposed.  However,  in  all  of  these  methods,  there  still  are  no 
general  standards,  which  could  be  used  as  a  design  criterion  of  video  enhancement  algorithms.  There  is  also  no  general 
unifying  theory  of  video  enhancement.  The  survey  of  available  techniques  is  based  on  the  existing  techniques  of  video 
enhancement,  which  can  be  classier  into  two  broad  categories:  spatial-based  domain  video  enhancement  and 
transform-based  domain  video  enhancement  [1,  9-14].  Spatial-based  domain  video  enhancement  operates  directly  on 
pixels.  The  main  advantage  of  spatial-based  domain  technique  is  that  they  are  conceptually  simple  to  understand,  and  the 
time  complexity  of  these  techniques  is  low  which  favors  real  time  implementations.  But  these  techniques  generally  lacks  in 
providing  adequate  robustness  and  imperceptibility  requirements.  A  survey  of  spatial-based  domain  enhancement 
techniques  can  be  found  in  [4,  15-18].  Transform  based  domain  video  enhancement  is  a  term  used  to  describe  the  analysis 
of  mathematical  Functions  or  signals  with  respect  to  frequency,  and  operate  directly  on  the  transform  coefficients  of  the 
image,  such  as  Fourier  transform,  discrete  wavelet  transforms  (DWT),  and  discrete  cosine  transforms  (DCT) 
[1,  13-14,  18-20].  The  basic  idea  in  using  this  technique  is  to  enhance  the  video  by  manipulating  the  transform  coefficients. 
The  advantage  of  transform-based  video  enhancement  include  (i)  Low  complexity  of  computations,  (ii)Ease  of  viewing 
and  manipulating  the  frequency  composition  of  the  image,  and  (iii)  the  easy  applicability  of  special  transformed  domain 
properties.  The  basic  limitations  including  (i)  it  cannot  simultaneously  enhance  all  parts  of  the  image  very  well,  and  (ii)  it 
is  difficult  to  automate  the  image  enhancement  procedure. 

LITERATURE  SURVEY 

Different  genetic  approaches  have  been  applied  for  images/videos  contrast  enhancement  [36].  [37]  Uses  a  local 
enhancement  technique.  Genetic  algorithms  are  meta-heuristic  optimization  techniques  based  on  natural  theory  and 
survival  of  the  fittest.  [32]  Uses  a  simple  chromosome  structure  and  genetic  operators  to  increase  the  visible  details  and 
contrast  of  low  illumination  images  especially  with  high  dynamic  range.  In  particular  the  enhancement  of  very  dark  and 
blurred  images  has  been  of  particular  interest  as  many  aspects  of  these  images  are  ambiguous  and  uncertain  [38]. 
The  method  based  on  transform  function  that  stretches  the  occupied  gray  scale  range  for  the  image  secondly  the 
transformation  function  is  optimized  using  genetic  algorithms  with  respect  to  the  test  image. 

Tone  mapping  is  another  approach  contrast  enhancement  technique.  In  this  method,  if  we  want  to  output  high 
dynamic  range  (HDR)  image  on  paper  or  on  a  display,  we  must  somehow  convert  the  wide  intensity  range  in  the  image  to 
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the  lower  range  supported  by  the  display  [39].  However,  most  LCD  or  CRT  displays  and  print-outs  have  low  dynamic 
range.  Tone  mapping  technique  used  in  image  processing  and  computer  graphics  to  map  asset  of  colors  to  another,  often 
approximate  the  appearance  of  high  dynamic  range  images  in  media  with  a  more  limited  dynamic  range.  Tone  mapping  is 
done  in  the  luminance  channel  only  and  in  logarithmic  scale.  It  is  used  to  convert  floating  point  radiance  map  into  8-bit 
representation  for  rendering  applications. 

The  two  main  aims  of  tone  mapping  algorithm:  preserving  image  details  and  providing  enough  absolute 
brightness  information  in  low  dynamic  range  tone  mapped  image.  The  existing  techniques  of  tone  mapping  can  be 
classified  into  categories:  global  tone  mapping  and  local  tone  mapping.  Global  tone  mapping  function  is  based  on 
logarithmic  compression  of  luminance.  It  is  a  need  to  map  the  large  range  into  a  range  that  can  be  displayed  given  a  HDR 
image  with  dynamic  range  spanning  many  orders  of  magnitude.  Simple  mapping  methods,  such  as  the  function  y  =  x=(x; 
y)  will  map  a  range  of  x  =  [0;  8)  to  the  range  x  =  [0;  1). 

This  method  is  considered  a  local  operator  since  the  operation  only  affects  pixel  values  in  an  image  individually 
on  a  pixel-by-pixel  basis  and  each  pixel  is  mapped  in  the  someway.  The  global  are  independent  of  local  spatial  context. 
It  performs  the  same  operation  on  each  pixel  and  don't  work  well  when  illumination  varies  locally.  The  simplest  tone 
reproduction  is  a  linear  mapping  which  scales  the  radiances  to  the  range  between  0  and  255.  The  logarithm  of  the  radiances 
is  taken  and  linearly  scaled  to  [0,  255].  Tone  mapping  algorithm  designed  for  high  contrast  images  is  widely  accepted. 
Multipass-based  technique  first  estimates  local  adaptation  level,  and  applies  simple  tone  mapping  function  to  it,  and  then 
puts  back  image  details.  The  approach  follows  functionality  of  human  visual  system 

(HVS)  without  attempting  to  construct  its  sophisticated  model.  The  definition  local  contrast  C  at  a  pixel  is  as 

follows. 

C(x,V)  =  L(x,y)/La(x.y)-l 

Where  L  is  pixel  luminance  and  La  is  local  adaptation  level,  which  we  take  to  be  just  an  average  luminance  over 
some  neighborhood  around  pixel  position  (x;  y). 

Gradient-based  domain  tone  mapping  algorithm  is  used  to  display  high  dynamic  range  video  sequences  in  low 
dynamic  range  devices.  [40]  Obtain  a  pixel  wise  motion  vector  field  and  incorporates  the  motion  information  into  the 
Poisson  equation.  Then,  by  attenuating  large  spatial  gradients,  the  algorithm  can  yield  a  high-quality  tone  mapping  result 
without  flickering  artifacts.  Tone  mapping  technique  can  adjust  image  or  video  content  for  optimum  contrast  visibility 
taking  into  account  ambient  illumination  and  display  characteristics. 
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Table  1:  Survey  of  Video  Enhancement 


""'  

--  1 

— . — ^ — 

52  

■ 

<■  — — — 

.  .. 

— *»—  . — 

~ 

2r  

 i ~ 

*  ^  ' — 

'^ZZZ — 
*r7  ^.,._"^.'J~ 

.  .— .  -,„  

LH 1  i  1 1 

v  • ,  ■ 

Br — L^lT'uJlJ.ot^- 

■  ;  L-" 

TV  r'T   ' :  -  ' 

'TT, . .  =T  L.L. 

-  [ —  -n-  -.1                >  .  -.  1. 

jii      -n~..— ir.i... 

J 1  — . . .  1  ■  . 
-.  ...  ,  , 

■  '■  ^.-.-l.l  

^HL— . 

_■  -  ; — 

t.  _A  *  -  *  - 
■ ,  — 

'.'T"n;r.i  -i/i 

— it: — n  

M   !'-!'  i'.'lL.   

hi .  aca — tss^s: 

13] 

.  ■■■  .  '" 

.UTT.TJ'i 

...      .  - 

BM*  A.   

High  dynamic  range  imaging  (HDRI  or  just  HDR)  is  a  set  of  techniques  that  allow  a  greater  dynamic  range  of 
luminances  between  the  lightest  and  darkest  areas  of  an  image  than  standard  digital  imaging  techniques  or  photographic 
methods.  This  wider  dynamic  range  allows  HDR  images  to  more  accurately  represent  the  wide  range  of  intensity  levels 
found  in  real  scenes,  ranging  from  direct  sunlight  to  faint  starlight  [15].  However,  the  dynamic  range  in  real-world 
environments 
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Figure  1:  Video  Frames 

Thus  far  exceeds  the  range  represented  in  8-bit  per-channel  texture  maps.  Extended  dynamic  range  can  be  reached 
by  combining  multiple  images  of  the  same  scene  taken  with  different,  known  exposure  times.  The  result  is  a  floating  point 
radiance  map  with  radiance  values  being  proportional  to  those  observed  in  the  real  scene.  The  two  main  sources  of  HDR 
imagery  are  computer  renderings  and  merging  of  multiple  photographs,  which  in  turn  are  known  as  low  dynamic  range 
(LDR)  [39]. 

PROPOSED  METHODOLOGY 
Proposed  Algorithm 

•  Select  an  AVI  video 

•  Check  for  its  Header. 

•  If  uncompressed  then  go  to  step  4  else  go  to  step  8. 

•  Split  video  into  frames  and  sound. 

•  Enhance  Frames  and  sound  (Optional). 

•  Create  video  from  enhance  frames. 
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•  Embed  sound  into  newly  created  video. 

•  stop 
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Figure  2:  Data  Flow  Diagram  (Proposed  System) 
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Figure  3:  Proposed  System  Architecture 

Image  Enhancement 
Algorithm 

•  Select  an  Image 

•  Read  Image  pixels. 

•  Modify  an  Image  pixels  components  (Red,  Green  and  Blue)  with  an  Equation 
Pnew=  j(Pr\\Pg\\Pb)  *  Ef  +  Biase  Eq  3.2.1 

•  If  Pnew>255  II  Pnew<0  then  discard  pixels  and  set  its  original  value  as  new  Pixel  value. 

•  Repeat  step  3  to  4  until  all  pixels  are  scanned. 

•  Stop 

The  quality  of  an  image  is  examined  by  objective  evaluation  as  well  as  subjective  evaluation.  For  subjective 
evaluation,  the  image  has  to  be  observed  by  a  human  expert.  But  the  human  visual  system  is  so  complicated  and  this  cannot 
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give  the  exact  quality  of  image.  There  is  various  metrics  used  for  objective  evaluation  of  an  image.  Some  of  them  are  mean 
square  error  (MSE),  root  mean  squared  error  (RMSE),  mean  absolute  error  (MAE)  and  peak  signal  to  noise  ratio  (PSNR). 

Mean  Square  Error  (MSE)  is  given  as 

Hek  Sqrwme  Emir  F.)  is  .prvraiit* 

.W3E  »  X  £ c  ^c  m'  **> Hv.nVf 

\1l-.u  i  ,V:i  •  i'  j---  F  •  i  mm    ■   ik  fin-.:  .■  . 

MAE  =  j£]£||3i<jw.,>t}--J|<i».«)| 

Peak  signal  to  noise  ratio  (PSNR)  is  defined  in  logarithmic  scale  in  db.  It  is  a  ratio  of  peak  signal  power  to  noise 
power.  Since  the  MSE  represents  the  noise  power  and  the  peak  signal  power,  the  PSNR  is  defined  as 

PSNR  =  IO*tog  10(1 /MSE) 

This  image  metric  is  used  for  evaluating  the  quality  of  a  filtered  image  and  thereby  the  capability  and  efficiency  of 
a  filtering  process.  In  addition  to  these  metrics,  universal  quality  index  (UQI)  is  extensively  used  to  evaluate  the  quality  of 
an  image  now-a-days.  Further,  some  parameters,  e.g.  method  noise  and  execution  time  are  also  used  in  literature  to 
evaluate  the  filtering  performance  of  a  filter  [14-15]. 

EXPERIMENT  RESULTS 

Table  2 


Sr.No 

Image  Size 

Result  Image  Size 

PSNR 

1 

100  X  100 

100  X  100 

70.57 

2 

150 X  150 

150 X  150 

75.2143 

3 

200  X  200 

200  X  200 

80.237 

4 

250  X  250 

250  X  250 

67.2382 

5 

300  X  300 

300  X  300 

76.2342 

Table  3 


Sr.No 

Image 
Name 

Size 

Mean 
Intensity 

Entropy 

Enhance 
Image 
Size 

Mean 
Intensity 

Entropy 

1 

1-jpg 

100x100 

0.5 

17.35 

100x100 

0.43 

17.62 

2 

2jpg 

150x150 

0.4 

18.24 

150x150 

0.478 

18.23 

3 

3jpg 

200x200 

0.5 

17.352 

200x200 

0.5 

17.987 

4 

4jpg 

250x250 

0.6 

18.234 

250x250 

0.6234 

18.464 

Figure  4:  Input  Video  Frame 
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Figure  5:  Output  Video  Frame 
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CONCLUSIONS 

This  proposed  video  enhancement  framework  consisting  of  Bilateral  Tone  adjustment  and  SWCE.  SWCE  method 
integrates  the  saliency  map  with  a  simple  contrast  enhancement,  and  also  performs  more  enhancements  in  regions  that 
humans  give  more  attention.  This  work  showed  that  SWCE  achieves  greater  performance  using  luminance  component. 
To  evaluate  the  enhancement  performance,  the  PSNR  value  was  used  to  measure  the  quality  of  enhancement.  This 
technique  will  also  prove  that  enhancing  the  quality  of  low-grade  video  surveillance  cameras. 
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